Papers with task-driven dialog systems
An Evaluation Dataset for Intent Classification and Out-of-Scope Prediction (D19-1)
Copied to clipboard
Stefan Larson, Anish Mahendran, Joseph J. Peper, Christopher Clarke, Andrew Lee, Parker Hill, Jonathan K. Kummerfeld, Kevin Leach, Michael A. Laurenzano, Lingjia Tang, Jason Mars
| Challenge: | Task-oriented dialog systems need to know when a query falls outside their range of supported intents. |
| Approach: | They propose a dataset that includes queries that are out-of-scope and 150 intent classes over 10 domains. |
| Outcome: | The proposed dataset includes queries that are out-of-scope, i.e., queries that do not fall into any of the system’s supported intents. |
Inconsistencies in Crowdsourced Slot-Filling Annotations: A Typology and Identification Methods (2020.coling-main)
Copied to clipboard
| Challenge: | Standard slot-filling models train or finetune on large datasets of carefully-annotated data that is domain specific. |
| Approach: | They propose automatic methods to identify inconsistencies in crowd-annotated data . a slot-filling model can extract the tokens "New York" as a TO LOCATION slot in a query . |
| Outcome: | The proposed methods reveal inconsistencies in data, though there is scope for improvement. |